An unbiased estimate for the mean of a {0, 1} random variable with relative error distribution independent of the mean

Say X1, X2, . . . are independent identically distributed Bernoulli random variables with mean p, so P(Xi = 1) = p and P(Xi = 0) = 1−p. Any estimate p̂ of p has relative error p̂/p−1. This paper builds a new estimate p̂ of p such that the relative error of the estimate does not depend in any way on the value of p. This allows the easy construction of exact confidence intervals for p of any desired level without needing any sort of limit or approximation. In addition, p̂ is unbiased. The expected number of Bernoulli draws used by the algorithm is at most 1 more than 1−p times the number of draws needed if the Central Limit Theorem held exactly. For ǫ and δ in (0, 1), to obtain an estimate where P(|p̂/p − 1| > ǫ) ≤ δ, the new algorithm takes on average at most 2ǫp ln(2δ)(1 − (4/3)ǫ) samples. It is also shown that any such algorithm that applies whenever p ≤ 1/2 requires at least (1/5)ǫ−2(1+2ǫ)(1−δ) ln((2−δ)δ−1)p−1 samples. The same algorithm can also be applied to estimate the mean of any random variable that falls in [0, 1]. Applications of this methodology include finding exact pvalues and estimating normalizing constants and Bayes’ Factors using acceptance/rejection.